{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img align=\"right\" style=\"max-width: 200px; height: auto\" src=\"cfds_logo.png\">\n",
    "\n",
    "###  Lab 03 - \"Moving Average Trading Strategies\"\n",
    "\n",
    "Chartered Financial Data Scientist (CFDS), Spring Term 2020"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this introductory lab, we create our first **financial data science process**. The main objective of this lab is to walk you through the general process of implementing and evaluating a simple **trend-following** trading strategy. To achieve this, we will follow the distinct process steps as outlined below:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img align=\"middle\" style=\"max-width: 600px; height: auto\" src=\"fds_process.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As always, pls. don't hesitate to ask all your questions either during the lab or send us an email (using our\n",
    "fds.ai email addresses)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lab Objectives:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After today's lab you should be able to:\n",
    "    \n",
    "> 1. Implement a **trend-following trading strategy** and apply it to distinct financial instruments.\n",
    "> 2. Convert the trading strategy results into **trade signals** to be used in backtest.\n",
    "> 3. Understand how to use the **python backtesting bt** library to backtest the implemented strategy.\n",
    "> 4. Interpret the backtests results using the distinct **backtest performance** measures."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we start let's watch a motivational video:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import YouTubeVideo\n",
    "# Nvidia GTC 2017: \"I Am AI\" Opening in Keynote\"\n",
    "# YouTubeVideo('SUNPrR4o5ZA', width=800, height=600)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup of the Analysis Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to import a couple of Python libraries that allow for data analysis and data visualization. In this lab will use the `Pandas`, `NumPy`, `BT` and the `Matplotlib` library. Let's import the libraries by the execution of the statements below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import python utility libraries\n",
    "import os as os\n",
    "import datetime as dt\n",
    "import itertools as it\n",
    "\n",
    "# import python data science libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# import the pandas financial data reader library\n",
    "import pandas_datareader as dr\n",
    "\n",
    "# import the matplotlib and seaborn visualization library\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install the Python `BT` backtesting library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install bt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upon successful installation let's import the Python `BT` backtesting library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import bt as bt # library to backtest trading signals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also set a couple of general plot parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set general plot parameters\n",
    "plt.style.use('seaborn')\n",
    "plt.rcParams['figure.figsize'] = [10, 5]\n",
    "plt.rcParams['figure.dpi']= 150"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Enable inline Jupyter notebook plotting:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppress potential warnings due to recent library enhancements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a **dataset** sub-folder that we will use to store the financial data downloaded:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not os.path.exists('./datasets'): os.makedirs('./datasets')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Acquire the Financial Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section of the lab notebook, we will aquire historic daily stock market data of the **\"International Business Machines\" (IBM)** corporation (ticker symbol: \"IBM\"). Thereby, we will utilize the `datareader` of the `Pandas` library that provides the ability to interface the `Yahoo` finance API. Let's first specify the start date and end date of the data download. We aim to download market price data starting from the **31.12.1990** until the **31.12.2017** to develop and evaluate a simple momentum trading strategy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set to start and end date of the data download\n",
    "start_date = dt.datetime(1990, 12, 31)\n",
    "end_date = dt.datetime(2017, 12, 31)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download the **daily** \"International Business Machines\" (IBM) market data of the defined timeframe using the `datareader`'s `Yahoo` finance API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# download ibm market data\n",
    "ibm_data = dr.data.DataReader('IBM', data_source='yahoo', start=start_date, end=end_date)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Pre-Process the Financial Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspect the top 10 records of the `IBM` data downloaded:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ibm_data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visually inspect the **adjusted closing price** of the downloaded `IBM` data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = [15, 5]\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "# plot ibm stock daily adjusted closing prices\n",
    "ax.plot(ibm_data.index, ibm_data['Adj Close'], color='#9b59b6')\n",
    "\n",
    "# rotate x-ticks\n",
    "for tick in ax.get_xticklabels():\n",
    "    tick.set_rotation(45)\n",
    "\n",
    "# set axis labels\n",
    "ax.set_xlabel('[time]', fontsize=10)\n",
    "ax.set_xlim([start_date, end_date])\n",
    "ax.set_ylabel('[adjusted closing price]', fontsize=10)\n",
    "\n",
    "# set plot title\n",
    "plt.title('International Business Machines Corporation (IBM) - Historical Stock Prices', fontsize=10);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save the downloaded `IBM` data to the local directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ibm_data.to_csv('./datasets/ibm_data_1990_2017_daily.csv', sep=';', encoding='utf-8')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Data Analysis - Moving Average Crossover Strategy Implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's implement a simple **Moving Average Crossover** trading strategy. In general, **crossover trading** refers to the idea that changes of market situations can be determined based on price “breakouts”. A crossover can be interpreted as another measure of a financial instruments momentum. In the past crossover signals have been extensively used to determine that it’s time to either buy or sell the underlying asset.\n",
    "\n",
    "The price crossover signals of a simple **Moving Average Crossover** trading strategy are triggered by the following events:\n",
    "\n",
    ">- Generate a **short** trading signal once the price of a financial instrument drops below the general price trend, e.g., 100-days moving average band (\"Sell Sign Crossover\", left image below). \n",
    ">- Generate a **long** trading signal once the price of a financial instrument exceeds the general price trend, e.g., 100-days moving average band (\"Buy Sign Crossover\", right image below)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img align=\"middle\" style=\"max-width: 800px; height: auto\" src=\"crossovertrading.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An enhancement of the **Moving Average Crossover** is to apply two moving averages to a chart: one long running moving average (e.g., a 200-days SMAV) and one short running moving average (e.g., 20-days SMAV). Once the short running moving average crosses above the long running moving average a **Buy** or **Long** signal is triggered, as it indicates that the trend is shifting up (this is known as a \"golden cross\"). On the other hand, when the short running moving average crosses below the long running moving average, a **Sell** or **Short** signal is triggered, as it indicates that the trend is shifting down (his is known as a \"dead/death cross\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start implementing this enhanced trading strategy by setting the distinct moving average window sizes that specify the number of historical daily adjusted closing prices of the IBM stock to be considered in the calculation of the rolling moving average:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_mav_days_15 = 15  # set \"fast\" short-running moving average indicator lookback, days = 15\n",
    "cross_mav_days_60 = 60  # set \"slow\" short-running moving average indicator lookback, days = 60\n",
    "cross_mav_days_200 = 200  # set \"trend\" long-running moving average indicator lookback, days = 200"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate the rolling moving averages of window sizes: 15 days, 50 days and 200 days. In general the **\"Simple Moving Average (SMAV)\"** of a financial instrument $i$ (e.g., a stock, commodity, fx-rate) is defined as the mean of the previous $n$ prices, formally denoted by: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$SMA_{i}(t)=\\frac{1}{n} \\sum_{k=0}^{n-1} p_{i}(t-k)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "were $t$ denotes the current point in time and $n$ the lookback."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can calculate the SMAV by just using the Pandas `rolling()` and `mean()`function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_mav_15 = pd.Series(ibm_data['Adj Close'].rolling(window = cross_mav_days_15).mean(), name = 'SMAV_15')\n",
    "cross_mav_60 = pd.Series(ibm_data['Adj Close'].rolling(window = cross_mav_days_60).mean(), name = 'SMAV_60')\n",
    "cross_mav_200 = pd.Series(ibm_data['Adj Close'].rolling(window = cross_mav_days_200).mean(), name = 'SMAV_200')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Merge the rolling moving average values with the original market data (adjusted closing prices):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_mav_ibm_data = ibm_data.join(cross_mav_15)\n",
    "cross_mav_ibm_data = cross_mav_ibm_data.join(cross_mav_60)\n",
    "cross_mav_ibm_data = cross_mav_ibm_data.join(cross_mav_200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspect and validate the daily adjusted closing prices of the IBM stock as well as the derived moving average values starting from the first obtained 200-day moving average market price:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_mav_ibm_data[['Adj Close', 'SMAV_15', 'SMAV_60', 'SMAV_200']].iloc[200:210]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the historical daily adjusted closing prices of the IBM stock (blue) as well as its 15 days (green), 60 days (red) as well as 200 days (yellow) rolling moving averages:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = [15, 5]\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "# plot adjusted closing prices and moving averages\n",
    "ax.plot(cross_mav_ibm_data['Adj Close'], lw=1.0, color='#9b59b6', label='Closing Prices (purple)')\n",
    "ax.plot(cross_mav_ibm_data['SMAV_15'], color='C1',lw=1.0, label='15-day MAV (green)')\n",
    "ax.plot(cross_mav_ibm_data['SMAV_60'], color='C1',lw=1.0, label='60-day MAV (red)')\n",
    "ax.plot(cross_mav_ibm_data['SMAV_200'], color='C4', lw=1.0, label='200-day MAV (yellow)')\n",
    "\n",
    "# rotate x-tick labels\n",
    "for tick in ax.get_xticklabels():\n",
    "    tick.set_rotation(45)\n",
    "    \n",
    "# set axis labels\n",
    "ax.set_xlabel('[time]', fontsize=10)\n",
    "ax.set_xlim([start_date, end_date])\n",
    "ax.set_ylabel('[market price]', fontsize=10)\n",
    "\n",
    "# set plot legend\n",
    "plt.legend(loc=\"upper left\", numpoints=1, fancybox=True)\n",
    "\n",
    "# set plot title\n",
    "plt.title('International Business Machines Corporation (IBM) - Daily Historical Stock Closing Prices', fontsize=10);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Moving Average Crossover Signal Generation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Derive trading signals from of two distinct moving average crossover trading strategy configurations. We will generate a **long-signal** (+1.0) for the time intervals where the fast moving averages are above the 200-day moving average. In addition we generate a **short-signal** (-1.0) for the time intervals where the fast moving averages are below the 200-day moving average:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create 'fast' trend-following signals\n",
    "cross_mav_ibm_data['SIGNAL_15'] = 0.0\n",
    "cross_mav_ibm_data.loc[cross_mav_ibm_data['SMAV_15'] > cross_mav_ibm_data['SMAV_200'], 'SIGNAL_15'] = 1.0\n",
    "cross_mav_ibm_data.loc[cross_mav_ibm_data['SMAV_15'] < cross_mav_ibm_data['SMAV_200'], 'SIGNAL_15'] = -1.0\n",
    "\n",
    "# create 'slow' trend-following signals\n",
    "cross_mav_ibm_data['SIGNAL_60'] = 0.0\n",
    "cross_mav_ibm_data.loc[cross_mav_ibm_data['SMAV_60'] > cross_mav_ibm_data['SMAV_200'], 'SIGNAL_60'] = 1.0\n",
    "cross_mav_ibm_data.loc[cross_mav_ibm_data['SMAV_60'] < cross_mav_ibm_data['SMAV_200'], 'SIGNAL_60'] = -1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition, let's also prepare a backtest of a **\"baseline\"** in terms of a simple **buy-and-hold** trading strategy for comparison purposes. Our buy-and-hold strategy sends a \"long\" (+1.0) signal for each time step: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_mav_ibm_data['SIGNAL_BASE'] = 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the trading signal data to be utilized in backtesting the long-/short-term moving-average trading strategy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert signals to Pandas DataFrame\n",
    "cross_mav_ibm_signal_data = pd.DataFrame(cross_mav_ibm_data[['SIGNAL_15', 'SIGNAL_60', 'SIGNAL_BASE']], columns=['SIGNAL_15', 'SIGNAL_60', 'SIGNAL_BASE'])\n",
    "\n",
    "# convert pandas DataFrame index to datatype: datetime\n",
    "cross_mav_ibm_signal_data = cross_mav_ibm_signal_data.set_index(pd.to_datetime(ibm_data.index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspect top 10 rows of the prepared trading signals:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_mav_ibm_signal_data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspect some of the exemplary signal deviations between the 15-days and 60-days crossover moving average trading strategies:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_mav_ibm_signal_data[cross_mav_ibm_signal_data['SIGNAL_15'] != cross_mav_ibm_signal_data['SIGNAL_60']].head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize the prepared trading signals:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = [15, 5]\n",
    "fig, ax = plt.subplots(ncols=1, nrows=3)\n",
    "\n",
    "ax[0].plot(cross_mav_ibm_signal_data['SIGNAL_15'], lw=1.0, color='C2', label='SMAV 16 (red)')\n",
    "ax[1].plot(cross_mav_ibm_signal_data['SIGNAL_60'], lw=1.0, color='C1', label='SMAV 60 (green)')\n",
    "ax[2].plot(cross_mav_ibm_signal_data['SIGNAL_BASE'], lw=1.0, color='C3', label='BASE (purple)')\n",
    "    \n",
    "# set axis labels\n",
    "plt.xlabel('[time]', fontsize=10)\n",
    "ax[0].set_xlim([start_date, end_date])\n",
    "ax[0].set_ylabel('[smav 15 signal]', fontsize=10)\n",
    "ax[1].set_xlim([start_date, end_date])\n",
    "ax[1].set_ylabel('[smav 60 signal]', fontsize=10)\n",
    "ax[2].set_xlim([start_date, end_date])\n",
    "ax[2].set_ylabel('[base signal]', fontsize=10)\n",
    "\n",
    "# rotate the x-axis labels\n",
    "for tick in ax[0].get_xticklabels():\n",
    "    tick.set_rotation(45)\n",
    "    \n",
    "for tick in ax[1].get_xticklabels():\n",
    "    tick.set_rotation(45)\n",
    "    \n",
    "for tick in ax[2].get_xticklabels():\n",
    "    tick.set_rotation(45)\n",
    "\n",
    "# set plot title\n",
    "ax[0].set_title('International Business Machines Corporation (IBM) - 15 days Crossover Moving Average Trading Signals', fontsize=10)\n",
    "ax[1].set_title('International Business Machines Corporation (IBM) - 60 days Crossover Moving Average Trading Signals', fontsize=10)\n",
    "ax[2].set_title('International Business Machines Corporation (IBM) - Baseline Buy and Hold Trading Signals', fontsize=10)\n",
    "\n",
    "# reset plot layout\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's determine the total number of **long-short signal changes** of the distinct trading strategies:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# signal changes of the 15-200 days moving average crossover trading strategy\n",
    "len(list(it.groupby(cross_mav_ibm_signal_data['SIGNAL_15'], lambda x: x > 0)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# signal changes of the 60-200 days moving average crossover trading strategy\n",
    "len(list(it.groupby(cross_mav_ibm_signal_data['SIGNAL_60'], lambda x: x > 0)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# signal changes of the baseline buy and hold trading strategy\n",
    "len(list(it.groupby(cross_mav_ibm_signal_data['SIGNAL_BASE'], lambda x: x > 0)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Moving Average Crossover Signal Backtest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the market data to be utilized in backtesting the crossover moving average trading strategy configurations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# extract the ibm stock closing prices\n",
    "ibm_market_data = pd.DataFrame(ibm_data['Adj Close'], columns=['Adj Close'])\n",
    "\n",
    "# rename the 'close' column to 'ibm' (since this is the column we want to allocate to in the backtest)\n",
    "ibm_market_data = ibm_market_data.rename(columns={'Adj Close': 'IBM'})\n",
    "\n",
    "# convert pandas DataFrame index to datatype: datetime\n",
    "ibm_market_data = ibm_market_data.set_index(pd.to_datetime(ibm_data.index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspect top 10 rows of the prepared market data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ibm_market_data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Implementing a simple Moving Average Trading Strategy by interfacing the Python `bt`'s Algo class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MovingAverageStrategy(bt.Algo):\n",
    "    \n",
    "    # inits the strategy\n",
    "    def __init__(self, signals):\n",
    "        \n",
    "        # set class signals\n",
    "        self.signals = signals\n",
    "        \n",
    "    # calss the trading strategy\n",
    "    def __call__(self, target):\n",
    "        \n",
    "        # case: current timestep in signals\n",
    "        if target.now in self.signals.index[1:]:\n",
    "            \n",
    "            # get actual signal\n",
    "            signal = self.signals[target.now]\n",
    "            \n",
    "            # set target weights according to signal\n",
    "            target.temp['weights'] = dict(IBM=signal)\n",
    "            \n",
    "        # return 'True' since we want to move on to the next timestep\n",
    "        return True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the moving average trading strategy backtest algorithm stack. \n",
    "\n",
    "**Note:** That in the Python `bt` library a trading strategy usually consists of a so-called **stack of algorithms**. For each timestep of our backtest timeframe, the `bt` library executes all algorithm of the stack in sequential order. Each moving average strategy we aim to design and backtest consists in total of three algorithms, briefly described in the following: \n",
    "\n",
    "> 1. `bt.algos.SelectAll()`: Selects all available stocks for trading except stock prices that correspond to NaN or 0.00.  \n",
    "> 2. `MovingAverageStrategy()`: Assigns the calculated signal in terms of a weight value to the IBM stock.\n",
    "> 3. `bt.algos.Rebalance()`: Rebalances the available capital based on the weights assigned to each stock."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the long-/short-term moving average trading strategy backtest algorithm stack: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_mav_ibm_strategy_15 = bt.Strategy(name='smav_15', algos=[bt.algos.SelectAll(), MovingAverageStrategy(cross_mav_ibm_signal_data['SIGNAL_15']), bt.algos.Rebalance()])\n",
    "cross_mav_ibm_strategy_60 = bt.Strategy(name='smav_60', algos=[bt.algos.SelectAll(), MovingAverageStrategy(cross_mav_ibm_signal_data['SIGNAL_60']), bt.algos.Rebalance()])\n",
    "cross_mav_ibm_strategy_base = bt.Strategy(name='base', algos=[bt.algos.SelectAll(), MovingAverageStrategy(cross_mav_ibm_signal_data['SIGNAL_BASE']), bt.algos.Rebalance()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now define the trading ('fees') commissions used in each rebalancing time-step of a backtest. To achieve this, the `bt` library expects a callable function that expects the following two parameters as an input:\n",
    "\n",
    "> - the 'quantity', denoted by `q`, of rebalanced assets at a backtest time-step;\n",
    "> - the 'price', denoted by `p`, of rebalanced assets at a backtest time-step.\n",
    "\n",
    "Let's implement such a callable function defining a trading fee of **1\\% (0.01)** per quantity of rebalanced asset (or a flat fee of **USD 5.00** per trade):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# init trading fees function\n",
    "def trading_fees_function(q, p):\n",
    "    \n",
    "    # calcluate trading fees (rebalanced-quantity * trading-fee)\n",
    "    fees = q * 0.01 # non-flat fee of 1% per quantity of rebalanced asset\n",
    "    # fees = 5.00 # flat fee of USD 5.00 per trade\n",
    "    \n",
    "    # return the total trading fees\n",
    "    return fees"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upon completion of defining the long-/sort-term moving average strategies let's now init the corresponding backtests using (1) both strategies as well as (2) the market data that we aim to evaluate during the backtest:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ibm_backtest_cross_mav_15 = bt.Backtest(strategy=cross_mav_ibm_strategy_15, data=ibm_market_data, name='ibm_backtest_smav_15', commissions=trading_fees_function, progress_bar=True)\n",
    "ibm_backtest_cross_mav_60 = bt.Backtest(strategy=cross_mav_ibm_strategy_60, data=ibm_market_data, name='ibm_backtest_smav_60', commissions=trading_fees_function, progress_bar=True)\n",
    "ibm_backtest_cross_mav_base = bt.Backtest(strategy=cross_mav_ibm_strategy_base, data=ibm_market_data, name='ibm_backtest_smav_base', commissions=trading_fees_function, progress_bar=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's run the backtest of both configurations of the crossover moving average strategy as well as the defined baseline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "backtest_results_ibm = bt.run(ibm_backtest_cross_mav_15, ibm_backtest_cross_mav_60, ibm_backtest_cross_mav_base)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspect the individual backtest results and performance measures:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "backtest_results_ibm.display()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Collect detailed backtest performance per timestep of the \"fast\" **15-200 days** crossover moving average strategy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "backtest_sma_15_ibm_details = ibm_backtest_cross_mav_15.strategy.prices.to_frame(name='Rel. EQUITY')\n",
    "backtest_sma_15_ibm_details['Abs. EQUITY'] = ibm_backtest_cross_mav_15.strategy.values # equity per timestep\n",
    "backtest_sma_15_ibm_details['CASH'] = ibm_backtest_cross_mav_15.strategy.cash # cash per timestep\n",
    "backtest_sma_15_ibm_details['POSITIONS'] = ibm_backtest_cross_mav_15.strategy.positions # positions per timestep\n",
    "backtest_sma_15_ibm_details['FEES'] = ibm_backtest_cross_mav_15.strategy.fees # trading fees per timestep"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspect detailed backtest results per timestep:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "backtest_sma_15_ibm_details.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize the monthly returns obtained by the **15-200 days** crossover moving average strategy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = [15, 10]\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "# plot heatmap of monthly returns generated by the strategy\n",
    "ax = sns.heatmap(ibm_backtest_cross_mav_15.stats.return_table, annot=True, cbar=True, vmin=-0.5, vmax=0.5)\n",
    "\n",
    "# set axis labels\n",
    "ax.set_xlabel('[month]', fontsize=10)\n",
    "ax.set_ylabel('[year]', fontsize=10)\n",
    "\n",
    "# set plot title\n",
    "ax.set_title('International Business Machines Corporation (IBM) - Monthly Returns SMAV 15-200 Strategy', fontsize=10);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Collect detailed backtest performance per timestep of the \"slow\" **60-200 days** crossover moving average strategy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "backtest_sma_60_ibm_details = ibm_backtest_cross_mav_60.strategy.prices.to_frame(name='Rel. EQUITY')\n",
    "backtest_sma_60_ibm_details['Abs. EQUITY'] = ibm_backtest_cross_mav_60.strategy.values # equity per timestep\n",
    "backtest_sma_60_ibm_details['CASH'] = ibm_backtest_cross_mav_60.strategy.cash # cash per timestep\n",
    "backtest_sma_60_ibm_details['POSITIONS'] = ibm_backtest_cross_mav_60.strategy.positions # positions per timestep\n",
    "backtest_sma_60_ibm_details['FEES'] = ibm_backtest_cross_mav_60.strategy.fees # fees per timestep"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspect detailed backtest results per timestep:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "backtest_sma_60_ibm_details.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize the monthly returns obtained by the **60-200 days** crossover moving average strategy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = [15, 10]\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "# plot heatmap of monthly returns generated by the strategy\n",
    "ax = sns.heatmap(ibm_backtest_cross_mav_60.stats.return_table, annot=True, cbar=True, vmin=-0.5, vmax=0.5)\n",
    "\n",
    "# set axis labels\n",
    "ax.set_xlabel('[month]', fontsize=10)\n",
    "ax.set_ylabel('[year]', fontsize=10)\n",
    "\n",
    "# set plot title\n",
    "ax.set_title('International Business Machines Corporation (IBM) - Monthly Returns SMAV 60-200 Strategy', fontsize=10);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Collect detailed backtest performance per timestep of the \"long only\" baseline strategy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "backtest_sma_base_ibm_details = ibm_backtest_cross_mav_base.strategy.prices.to_frame(name='Rel. EQUITY')\n",
    "backtest_sma_base_ibm_details['Abs. EQUITY'] = ibm_backtest_cross_mav_base.strategy.values # equity per timestep\n",
    "backtest_sma_base_ibm_details['CASH'] = ibm_backtest_cross_mav_base.strategy.cash # cash per timestep\n",
    "backtest_sma_base_ibm_details['POSITIONS'] = ibm_backtest_cross_mav_base.strategy.positions # positions per timestep\n",
    "backtest_sma_base_ibm_details['FEES'] = ibm_backtest_cross_mav_base.strategy.fees # fees per timestep"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspect detailed backtest results per timestep:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "backtest_sma_base_ibm_details.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = [15, 10]\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "# plot heatmap of monthly returns generated by the strategy\n",
    "ax = sns.heatmap(ibm_backtest_cross_mav_base.stats.return_table, annot=True, cbar=True, vmin=-0.5, vmax=0.5)\n",
    "\n",
    "# set axis labels\n",
    "ax.set_xlabel('[month]', fontsize=10)\n",
    "ax.set_ylabel('[year]', fontsize=10)\n",
    "\n",
    "# set plot title\n",
    "ax.set_title('International Business Machines Corporation (IBM) - Monthly Returns SMAV Baseline', fontsize=10);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize each strategie's backtest equity progression over time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = [15, 5]\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "# plot equity progression of the distinct trading strategies\n",
    "ax.plot(backtest_sma_15_ibm_details['Rel. EQUITY'], color='C2',lw=1.0, label='15-day SMAV strategy (red)')\n",
    "ax.plot(backtest_sma_60_ibm_details['Rel. EQUITY'], color='C1',lw=1.0, label='60-day SMAV strategy (green)')\n",
    "ax.plot(backtest_sma_base_ibm_details['Rel. EQUITY'], color='C3',lw=1.0, label='Base SMAV strategy (purple)')\n",
    "\n",
    "# rotate x-tick labels\n",
    "for tick in ax.get_xticklabels():\n",
    "    tick.set_rotation(45)\n",
    "    \n",
    "# set axis labels\n",
    "ax.set_xlabel('[time]', fontsize=10)\n",
    "ax.set_xlim([start_date, end_date])\n",
    "ax.set_ylabel('[rel. equity]', fontsize=10)\n",
    "\n",
    "# set plot legend\n",
    "plt.legend(loc=\"upper left\", numpoints=1, fancybox=True)\n",
    "\n",
    "# set plot title\n",
    "plt.title('International Business Machines Corporation (IBM) - Backtest % Equity Progression', fontsize=10);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab Exercises:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We recommend you to try the following exercises as part of the lab:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**1. Evaluation of distinct daily moving average parameters.**\n",
    "\n",
    "> Evaluate the simple crossover moving average trading strategy using distinct moving average look-backs, e.g., 10 days, 30 days, 50 days, 200 days and 300 days. Compare the performance of the lookback parametrizations in terms of total-return, equity progression and yearly sharpe-ratio. Gain an intuition about the years in which the strategy didn't perform well and the potential reason for the poor performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ***************************************************\n",
    "# INSERT YOUR CODE HERE\n",
    "# ***************************************************"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**2. Evaluation of distinct trading frequencies.** \n",
    "\n",
    ">Decreasing the lookback in calculating the moving average may results in a significant increase of the corresponding trading frequency (referred to as 'signal changes', and defined as a change of the trading signal from -1 to 1 and vice versa). Calculate the number of signal changes for each of the distinct lookback parametrizations evaluated in the first exercise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ***************************************************\n",
    "# INSERT YOUR CODE HERE\n",
    "# ***************************************************"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3. Consideration of trading commissions and impact on strategy performance.** \n",
    "\n",
    ">Run the backtest of the lookback parametrizations evaluated in exercise 1. but change the commission per trade. Set the trading commission of each backtest to (i) 10 USD per trade and (ii) 1% of the price per share. Determine the impact of such a trading fee on the performance of each strategy in terms of total-return, equity progression and yearly sharpe-ratio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ***************************************************\n",
    "# INSERT YOUR CODE HERE\n",
    "# ***************************************************"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**4. Optimization of the simple crossover moving average trading strategy parameters.**\n",
    "\n",
    ">The lab notebook backtest results obtained for the simple crossover moving average trading strategy reveals that we didn't identify a well-performing strategy parametrization yet. Grid search the parameter space of the strategy to determine a parametrization that results in a positive total-return within the time interval 30.12.1990 until 31.12.2017."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ***************************************************\n",
    "# INSERT YOUR CODE HERE\n",
    "# ***************************************************"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**5. Implementation of a long-/short-term exponential moving average trading strategy.**\n",
    "\n",
    ">A potential further enhancement of the simple crossover moving average trading strategy lies in the application of exponentially weighted moving averages. Adapt the crossover trading strategy of this notebook using an exponential moving average instead of the equally weighted moving average (hint: you may want to use the `ewm()` of the Pandas library). Evaluate the performance of the exponential moving average trading strategy using distinct alpha parameters, e.g.: 0.2, 0.5 and 0.8. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ***************************************************\n",
    "# INSERT YOUR CODE HERE\n",
    "# ***************************************************"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab Summary:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lab, a step by step implementation and backtest of a basic **moving average trading strategy** using the Python programming language is presented. The implemented strategy trades a specific financial instrument based on its adjusted closing price trend. The degree of success of the implemented strategy is evaluated based os its backtest results with particular focus on (1) the strategy's **total return** as well as (2) its **equity progression** over time. The code provided in this lab provides a blueprint to develop and backtest more complex trading strategies. It furthermore can be tailored to be applied for momentum trading of other financial instruments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may want to execute the content of your lab outside of the Jupyter notebook environment, e.g. on a compute node or a server. The cell below converts the lab notebook into a standalone and executable python script. Pls. note that to convert the notebook, you need to install Python's **nbconvert** library and its extensions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# installing the nbconvert library\n",
    "!pip3 install nbconvert\n",
    "!pip3 install jupyter_contrib_nbextensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now convert the Jupyter notebook into a plain Python script:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!jupyter nbconvert --to script cfds_colab_03.ipynb"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
